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RESEARCH  ON  COMPLEX  DECISION  MAKING,  KNOWLEDGE  REPRESENTATION,  AND  THE 
TRANSFER  OF  EXISTING  SYSTEMS  TO  ARI 


EXECUTIVE  SUMMARY 


This  report  briefly  describes  the  results  of  a  program  of  research  that 
focused  on  the  conversion  of  a  new  technology  into  a  form  that  is  facile  and 
generally  transferable  using  a  network  of  microprocessors.  When  the  contract 
was  awarded,  the  choice  (at  that  time  a  very  reasonable  choice)  of  hardware 
was  the  8-bit  microprocessor  and  (standard  to  ARI  and  other  institutions)  the 
Apple  II  microprocessor,  augmented  by  standard  peripheral  equipment. - 

The  program  suites,  appropriate  to  these  machines  and  a  Corvus  20 
Megabyte  Disc,  have  been  delivered,  as  originally  specified.  It  is  true  a  16- 
or  32-bit  technology  would  have  been  a  better  choice  since  obvious  limitations 
and  difficulties  crop  up  with  a  smaller  address  space.  However,  the 
networking  paradigm  remains  viable  and  may  only  be  replaced  by  a  multitasking 
system  with  very  sophisticated  display  and  control  capabilities  to  exhibit  the 
power  of  the  systems  to  any  user  and  render  the  computing  organization 
transparent,  as  is  required. 

The  deliverables  so  far  noted  are  CASTE  (Course  Assembly  System  and 
Tutorial  Environment),  THOUGHTSTICKER,  and  the  Team  Decision  System  (TDS) , 
together  with  manually  and  computer-administered  versions  of  a  test  for 
learning  innovative  and  conceptual  style,  SPC2. 

In  addition,  documentation,  source  code,  and  various  other  documents  are 
deliverable  products. 

While  the  program  emphasized  the  transfer  of  a  rendered- transportable- 
technology,  it  encouraged  the  dissemination  of  knowledge  and  further  research. 
Both  of  these  objectives  have  been  satisfied  to  the  extent  required  in  the 
Annexes  to  this  report. 
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Outline 


This  report  is  organised  as  -follows: 


Section  2.  is  an  introduction  to  and  an 
overview  o-f  the  work  done.  It  provides 
the  rationale  -for  integrating  the  various  ’ 
products. 

Section  3.  deals  with  THOUGHTST ICf-'Er'  1.0 
Section  4.  deals  with  TDS  l.C 
Section  5.  deals  with  CASTE  1.0 
Section  6.  deals  with  SF'C2 

Section  7.  deals  with  rules  -for  authoring 
which  (Te  st  be  respected  i  f  the  proper 
pei  son/machi  ne/i  nter-f  aces  are  to  function 
sati  s-f  acto-i  I  v  given  screen  display  and  a 
-few  other  limitations  of  C/CASTE/1 . 0. 
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RESEARCH  ON  COMPLEX  DECISION  MAKING,  KNOWLEDGE  REPRESENTATION, 
AND  THE  TRANSFER  OF  EXISTING  SYSTEMS  TO  ARI 


Over  vi  ew  anti  I  n  t roduc  t  i  on 


Over  a  period  of  about  7  —  6  years  various  human  machine 
systems  were  desioned  and  impi emented  under  the  whole  or  part 
support  of  the  ARI,  through  the  European  office,  to  System 
Research  Ltd..  U.K.  (the  tieg-ee  of  sponsorship  depending  upon 
the  system  in  question).  All  o-‘  them  arc  founded  in 
Conversation  Theory  and  its  protolocic  Lo  (described  in 
Annexe  1.;  an  ECTJ  invited  review  paper)  and  in  one  of  the 
products;  a  "Primer"  on  the  background. 

One  of  these  systems,  CASTE,  is  a  sophisticated  tutorial 
environment.  It  provides,  in  the  first  place,  an  accurate 
and  e,  te-ral i ssd  recced  of  how  I  earners  acquire  concepts 
which  are  represented .  together  with  their  relations,  in  a  so 
called  "prtai  Imr-nt  mesh " .  The  learna.bie  concepts  represented 
j.n  the  mesh  consist  minimally  of  one  (often  several) 
e:-:ece.t.-:..LI  e  models:-  a  description  or  behaviour  (obtainable  by 
erecuii  no  a  model  and  a.  reference  name  such  a  s  "Vessel"  or 
"F rifei-Llv  C  t  -nt  v  >•  "  or  "Country  Y“  or  "Routes  from  >  to  V" 
or  "Commander  of  Platoon",. 


ms.v  be 


(a  ■  on«  cr  no  re  compute."  programs  acting 
upai'i  a  specified  domai r, 

:  b  >  a  war  re  ocui cment  '-’or  e,  amcle.  a 
c  o  .mauler  t  r  a.  c  t  d  laboratory  -tor  p  e  r  +  a  r  m  i  r.  q 
statistical  or  physical  experi  mer.t  s;  ; 
t  ie  ecuic  mer-t  must  be  able  tc  trace, 
-■  vi'.'.  udh  a.r.d  return  a.  numer  i  c  al  i  ndax  cf 
tu cer  p e r  f  o r n ance . 


-.  o  >  a  sari  cs  of  conditions  and 
ir, struct  ions  that  determi  ne  a  role,  for 
example,  the  rcle  of  "Commander"  in  such 
a  wav  that  enactments  car.  be  evaluated  bv 
a  person  (usually  a  group  of  othe" 
learners,  provided  that  certain  essential 
restrictions  built  into  a'  program  are 
satisfied)  and  the  evaluation  returned  in 
numerical  form  to  CASTE  (for  example,  by 
measures  or.  a  repertory  grid  scheme). 


In  all 
model  must 


cases  (  and 
be  able,  on 


( a )  .  •  b  >  ,  ( c  ) 

CASTE  demand. 


are  not  exhaustive'-  ,  the 
to  dive  e  demonstration 


1 


of  haw  the  concept  is  replied  '"to  make  or  do  or  recognise 
some  objective)  and.  conversely,  to  accept  and  to  evaluate  an 
explanation  -from  a  learner  and  sc  to  give  CASTE  an  indication 
of  how  the  learner  uses  or  understands  the  concept  in 
quest  1  on . 

As  noted  alreadv,  concents  are  relates  in  a  "mesh".  This 
mesh  (also  known  as  an  “Lp  statement")  answers  the  whv 
question,  in  contrast  to  the  how  question  ie.  it  gives  an 
answer  to  the  question  “why  did  you  use  that  method  or  give 
that  e,. o  1  anat  1  on “  or,  with  equal  significance  "from  what 
other  concept  can  the  concept  that  a  learner  is  coaling  with 
be  derived."  Such  entailed  derivations  specify  how  one 
concept  model  (which  is  executable,  which  works)  can  be  built 
out  of  other  concept  models.  If  and  only  if  a  learner  can 
q i re  satisfactory  replies  to  the  what  and  how  questions  is  a 
scared  concent  said  to  bs  understood  and  is,  on  empirical 
grounds  permanent! y  resilient. 

Usual  I v  there  are  many  possible  derivations  soeci feed  i r 
an  entail mant  mesh,  structured,  for  examcie.  by  a  group  c-f 
e  certs.  F-u-tner,  the  e  is  no  r sason  to  limit  the  number  of 
shared  conceocs  wh i c *•  might  be  invented  and  which  do  wo-~r  , 


Co- .sequent  1  •/,  C  '2"Z  cor, tains  not  on  -  v  a  facil.t,  for 
reco-dir  c  che  process  o-‘  learning  i  n  great  cetail.  It  also 
has  uwtthori  no  f  as  i  1  i  t  i  os  tna.t  are  at  the  learners  ci soesa  3 
previ  dec  no  pari  of  an  or:  qi  nail  >  seti  sf  act  or  y  mesh  is 
csieted.  in  i- he  war  a  most  can  and.  in  pract.ce,  does 

eve  I  ve,  In  prr.ncioie.  CASTE  could  be  initialised  with  a 

bl mesh  (si  though  this  expedient  is  no:  usual  I v 
recommended  except  in  special-  appl icationsi s  for  example , 
where  learners  ha\s  or- or  knowledge  of  a  comain  (t 
instance-,  of  how;  to  manouvsr  or  allocate  resources  ir,  an 
uncarts: r  si  dual: on) . 


Finallv,  HAFT  E  is  c-'  ten  augmented  by  specific  tutorial 
aide.  It  has  been  found  in  numerous  experiments  that 

1  earners  have.  ind:  vi dually  preferred  strauooies  of  leai  nine, 
suc:h  as  follow!  no  one  route  assiduously  or,  on  the  other 
hand .  bui 2  di no  up  a  global  picture  and  completing  detailed 
task  accomplishment  later.  Again,  some  learners  e-e  able  to 
use  exact  analogies;  to  use. valid  qgneral  i  sa.ti  or.  s  tc  learn 
exemplars,  or  conversely.  to  move  from  exemplars  to  a 
aener  al i set i on . 


Such  tendencies.  whether  manifest  over  many  learning 
situations  or  con  sex t  specific,  come  out  as  one  component  of 
a  learning  'ecord.  Hence,  one  tutorial  heuristic  employs 
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this  1  earning  strategy  int or mat i cn  to  recommend  paths  that 
are  likely  to  render  subseuuent  learning  effecui\e  -for  the 
learner  in  question:  in  the  limit  case,  to  dictate  that 
certain  paths  are  taken  (this,  however,  is  seldom 
recommended;  some  measure  of  choice  is  needed  i  f  only  because 
these  learner  propensities  or  styles  are,  in  general .  context 
dependent ) . 

Another  readily  1  mu 1 emented  tutorial  heuristic  stems  from 
work  done  by  my  team,  20  or  25  years  ago,  on  compie;'  adaptive 
training  systems  which  matched  task  difficulty  to  proficiency 
in  order  to  achieve  a  balanced  learning  condition. 
Translated  into  CASTE  terms,  this  heuristic  ammounts  to 
uncertainty  regulation;  maintaining  enough  degree  of  choice, 
but  not  too  much  so  that  both  overload  and  boredom  are 
avoided.  The  balance  point,  for  a  particular  learner,  is 
again  set  by'  data  from  the  historical  record. 

CASTE,  used  in  a  tutorial  mode,  provides  (or  is 
initialised)  with  a  rsprssentaticn  of  shared  concepts 
(minimal  1 y  specified  above).  These  concepts  are  inscribed  in 
an  entai  1  merit  mesh  whi  in  also  consists  in  relations  that 
connect  them  together  and  the  entire  mesh  is  open  to  non 
deleting  evolution  (by  further  concepts  and/or  relations). 
There-  is  a  precise  sense  (as  noted  in  the  referenced  annexes) 
in  which: 

(a;  All  the  relations  or  "en tai 2  rent =,  ' 
represent  iooical  coherence  of  the  "< 
process ,  product  >  or  Execution  of 
Model  (s> f  ,  Product  (descr i pti or  cr 
D  s  n  a  v  i  o  u "  )  ’  "  p  air?  r.  i  n :  m  v  1  1  v  t  e  p  r  e  s  e  n  ting 
sha<  sd  com  tapis  a  no 

(b  >  Tne  Model  (Process)  aspect  of  a 
shared  concept  crystal  isss  the  answer's  to 
i.  how  1  1  bv  what  method")  question  in  a 
con versa ci on  throuah  CAS  It.  betweer 
par ti ci cants  m  the  conversation .  The 
products  answer  what  questions  ("what  is 
the  conversation  about")  and  tne 
relations  between  concepts  answer  tne 
"why"  questions  ("why  did  you  use  this 
method  and  I  use  that  method"). 

(z)  Hence,  a.  mesh  is  a  representation  zx 
knowables  and  an  open-to— mesh  evolution 
theory'  about  what  may  be  known. 

Between  1974  and  1975  it  became  clear  that  a  mesh  of  this 
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tvoe  is  more  thar  an  inscription.  father,  it  embodies  a 
caicclus.  the  protoicgic,  Lp ,  of  conversation  theory  and  is 
mam  pul  ab  i  e.  Lor  ex  amp  1  s  .  inf  erences  can  be  (and.  in  real 
systems,  are)  made  to  propose  further  coherencies,  subject  to 
the  overridinc  rule  that  original ly  asserted  distinctions 
between  concerts  are  neither  obscured  nor  denies. 


The  next  product,  THOUGHTST ICKER,  operates  in  the  CASTE 
environmant  to  perform  basic  protolog: cal  manipulations  upon 
the  mesh.  The  THOUGHTST ICKER  operations  assume  that  anv 
shared  concept  i_s  such  a  thine;  either,  it  has  an  executable 
model  or  one  will  be  provided  using  the  "Configure"  Routine 
which  is  continually  available  in  CASTE;  taken  together, 
T HDL-GHTST I CKER  and  the  "Confiaure"  Routines  f  orm,  an  authori  no 
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chat  certain  concepts  stici 
r  from  the  others  (Fig,  1.) 

■  s  o'  a  concept  are  possible 
p-'-i  the  concept  in  mind  and 
2.  (using  coherencies  that 
di  sti  rcti  or.  is  lost  both 


special  unidirectional  analogy;  all  of 


the  THOUGHTST ICKER  operations  carried  out  on  a  mesh.  Of 
these,  two  are  directional  unfoldments  of  a  mesh  to  produce 
ordered  structures,  from  a  specified  point  of  view 
(representing  learning  strategies  or  strategies  of  planning 
and  action).  The  first  of  the  two  is  called  a  "Prune" 


operation  (Fig-  3)  and  the  next  is  a  selective  pruning  (Fig. 
4)  which  dissects  the  class  of  all  learning  or  action 
sequences,  as  the  case  may  be,  into  specific  conditional  and 
program  like  structures  that  are  all  open  to  execution.  Some 
caution  is  needed  insofar  as  the  execution  m$  be  initiated 
after  several  stages  as  indicated  below  (in  the  case  of 
general i sati ons) .  Hence,  it  is  better  to  consider  "Prune" 
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Collective  Coherences 
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Fig.  4 


Sel prune  (0,  T.ft)  Selprune  (1,  T,A)  Selprune  (2,  T.ft) 


Genova  Contravention 
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and  "Selprune"  conjoined  to  execution  in  terms  of  an 
iterative  operation  "Expand"  represented  and  implemented  by 
(the  n  -fold  iteration)  of  a  sequence  like. 

"Prune  (-from  some  concept);  ((Select  (Selprune)); 
(Execute  to  Model)";  that  is,  although  any  shared  concept  has 
an  executable  model,  this  model  may  be  and  o-ften  is  a  mesh, 
the  shared  concepts  o-f  which  may  either  point  to  other  meshes 
or  directly  and  with  equal  legitimacy  to  models. 

The  next  THOUGHTS! I CKER  operation  is  "saturation"  (Fig. 
5),  which  is  an  inference  rule,  appliable  if  and  only  if  a 
sufficient  number  of  distributive  coherencies  exist.  If  so, 
then  "saturation"  proposes  further  coherencies  that  do  not 
loose  already  asserted  distinction  between  shared  concepts 
that  are  Lp  legal.  The  author  or  team  of  authors  may,  of 
course,  deny  or  accept  these  proposals,  in  which  case  the 
number  of  proposals  is  greatly  reduced. 

"Pruning"  and  "Selective  Pruning"  are  forms  of  unfoldment 
and,  thus,  of  Lp  legality  criteria.  If  a  mesh  is  "prunable", 
then  it  is  legal;  if  not,  then  it  is  either  modified  or 
transf  ormed . 


The  cr 
the  "Rule 
Olvmpia  an 
or  so  in 
those  days 
The  least 
r esc  1 ut i on 
resol ut 1  on 
act l ons. 
analogy  pr 
for  general 


ucial  point  is  loss  of  distincti 
of  Genova",  named  after  Vittorio 
d  Renaldo  Sana  (of  whom  Vittorio 
my  own  laboratory;  Olympia  visi 
,  Renaldo  Sana.  headed  the  CNR 
elaborate  case  of  conceptual 
is  shown  (in  condensed  form)  in 
ammounts  to  anal ogy  creating 
These  are  shown  (as  outlines)  i 
eductions)  and  in  Fig.  B  (for  gen 
isatian  resolution  in  Fiq.  9. 


on  as  manifest  in 
Midora  and  George 
worked  for  a  year 
ted  often,  and  in 
group  in  Genova). 

conflict  and  its 
Fig.  6.  Conflict 
and  generalising 
n  Fig.  7  (various 
eralisations'  and 


It  may  be  added  that  one  entail ment  mesh  can  be  and 
usually  is  quite  large.  In  addition,  it  is  often  necessary 
to  deal  with  distinct  meshes  related  by  analogy  and/or 
gener al i sat i on ,  shown  in  Fig.  10  and  in  Fig.  11.  So, 
finally,  there  is  an  interesting  situation  in  which  the 
combination  and  continued,  iterative,  application  of  these  Lp 
operations  lead  to  an  emulation  of  personal  concept 
production  and  reproduction  in  Conversation  Theory. 


The  notion  of  emulating  personal 
spelled  out  in  Section  9.  Theoreti 
concurrent  algorithmic  spec i f i cat i on 
reapraisal  and  enhancement  of  the 
Conversation  Theory,  C.T. 


concept  production  is 
cal  work  has  led  to  a 
of  a  considerable 
protologic,  Lp ,  of 
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In  order  to  avoid  repetition  (some  readers  are  already 
■familiar  with  Lp  and  its  C.T.  basis.  some  are  not)  the 
Appendices  include  a  recent  invited  review  paper  so  that 
readers  who  are  not  happy  about  the  Lp  and  C.T.  postulates 
(whether  in  accord  with  them  or  not)  can  consult  it  as  a 
pre-face  to  Section  9. 
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THOUGHTST I CK'ER  Version  1.0 


As  noted  already  an  entailment  mesh  is  a  class  of 
expressions  in  the  protologic,  Lp,  of  C. T.  and  specific 
assumptions  are  made  (notably,  a  model  is  an  integral  part  Of 
any  shared  concept  or  "topic"  in  the  entailment  mesh). 
Unless  the  assumed  requirements  are  satisfied,  Lp  Operations 
carried  out  on  topic  names  and  relations  are  arbitrary  string 
manipulations.  On  the  other  hand,  if  these  requirements  are 
satisfied,  then  operations  upon  topic  names  are  meaningful. 
The  use  of  THOUGHTST 1CKER  1.0,  which  treats  topic  names  as 
though  they  designated  shared  concepts  is  thus  only  sensible 
if  names  said  to  indicate  concepts  do,  in  fact,  designate  a 
sufficient  exteri or i sati on  (minimally  an  executable  model  and 
a  product)  of  a  shared  concept  ie.  if  the  what,  how  and  why 
questions  of  Section  2  have  been  posed  and  answered 
sat i  5-factor i  1  y . 

From  a  users  point  of  view  it  would  be  obsessive  to 
require  that  all  models  exist  when  THOUGHTST I CKER  1.0  is 
used.  It  is  sufficient  to  be  certain  that  they  will  exist 
before  the  mesh  is  used  by  a  learner  or  another  author.  In 
practice,  some  authors  start  with  many  models  fully 
specified;  others  from  an  overview  of  a  subject  matter  or 
skill  and  the  rational  conviction  that  models  (perhaps  hazily 
in  mind)  can  be  refined  and  implemented.  But,  in  any  case, 
authors  must  be  aware  of  those  requirements.  Lacking  the 
awareness,  the  use  of  THOUGHTSTICKER  1.0  is  logically  unsound 
and  often  counterproduct i ve  as  experience  has  amply  and 
frequently  shown. 

It  is  worth  exemplifying  this  point,  in  order  to  give  it 
the  emphasis  needed.  To  do  so,  take  a  mathemati cal  1 y  precise 
subject  matter;  conveniently  part  of  "statistics",  where 
there  is  no  serious  ambiguity  at  all  (this  is  not  to  suggest 
that  THOUGHTSTICKER  1.0,  let  alone  Lp ,  is  restricted  to  such 
subject  matter  domains). 

Let  V  =  Variation 

S  =  Scatter  (of  a  numerical  quantity 
about  an  average  value) 

M  =  Mean  of  n  quantities  U 
D  =  Distance  (numer i cal  1 y )  from  mean 
D2  =  Squared  Distance 
Z  =  Modulus  (positive  value)  of  D  or 
B  (below) 

B  =  Numerical  distance  (from  average 
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val ue) 

A  =  Average  (or  median  or  mean) 
Suppose , 

(a)  V  is  derived,  by  entailment 
•from  M,  D  and  D2 

(b)  V  is  derived,  by  entailment 
from  A,  B,  Z  and  S 

Given  (a)  one  possibility  is 
V  =  Sum  D2/n 
D2  =  D  Squared 
D  =  U  -  M 

Given  (b)  V  =  one  kind  of  S, 
and  from  average  median  or  mean 
(S,Restri cted  by  R.M.S. 

distance) 

S  =  Sum  Z/n 
Z  =  Modulus  U  -  A 


Clearly,  both  of  these  and  many  others  can  be  modelled 
and  the  model  of  V  obtained  as  something  generating  either  5 
or  (the  usual)  variance.  If  we  wish  to  obtain  standard 
deviation  than  any  derivation  must  have  a  form  that  makes 
sense  of  only  variance  by  pointing  out,  for  example,  the 
restrictions  of  the  more  usual  (but  less  intuitive)  variance. 
However,  both  of  the  derivations  of  V  make  sense  and  (at  the 
cost  of  less  convenient  forms  of  di stri buti on )  have 
statistical  merit.  The  simple  derivation  of  the  standard 
deviation  from  V  points  to  the  rationale  of  adopting  variance 
as  an  index  of  variation,  V,  and  also  for  certain  any  other 
indices  of  variability,  for  example,  several  that  are  used  in 
non  -  linear  domains  and,  as  a  matter  of  fact,  some  non  - 
parametric  piecewise  continuous  or  only  ordered  statistical 
methods.  The  fact  that  this  rationale  is  often  omitted 
indicates  the  strength  of  Lp  and  of  THOUGHTSTICK'ER  1.0  if 
properly  employed.  It  is  easy  enough  to  cite  inefficiencies 
or  even  disasters  that  might  have  been  avoided  if 
responsible,  trained  personnel  had  thought,  rather  than 
reacted  ,  to  a  superior's  requirement  "Tell  me,  what's  the 
variability  of  those  figures"  by  the  tacitly  acceptable  "That 
guy  wants  a  variance"  which  was  dul y  ,cal cul ated  on  a  pocket 
calculator  and  phoned  back. 

With  this  essential  preamble,  THOUGHTSTICKER  1.0  is  able, 
apart  from  display,  mesh  recording  and  the  like,  to  perform 
the  following  Lp  operations  reliably. 
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(1)  Pruning  -from  any  specified  topic 

(2)  Selective  pruning  of  a  given  pruning 

(3)  Retrieving  meshes 

(4)  Saturation 

(5)  Rule  of  Genova  contravention  or  lack 
of  asserted  distinction 

(6)  Various  dependent  operations  such  as 
topic  listing  and  cluster  listing, 
alphabetic  display,  level  display. 


These  operations  are  described  in  Section  2  using  the 
same  framework.  But,  preferably  to  ensure  that  the 
emphasised  conditions  are  satisfied;  they  should  be  carried 
out  in  the  environment  of  CASTE  1.0  where  it  is  possible  to 
add  currently  specified  algorithms  as  programmed 
implementations  that  greatly  enhance  the  value  of  the  system. 
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TDS,  the  Team  Decision  System,  implemented  as  TDS  1.0 


The  purpose  of  the  team  decision  system  is  to  provide  a 
very  general  C3  training  and  operation  simulating  system, 
various  con-figurations  were  developed,  tested  and  used  during 
the  middle  and  late  1970's  and  this  development,  at  the 
laboratory  o-f  System  Research  Ltd.,  Richmond,  Surrey,  U.K. 
was  in  large  measure  tailored  to  incorporate  the  findings  of 
4  Richmond  Conferences,  on  complex  decision  making  in  C3, 
sponsored  by  the  ARI  and  organised  by  System  Research  in 
consultation  with  the  European  office  of  ARI  and  the  U.S. 
ARI.  The  findings  which  came  from  these  conferences  were  of 
immense  Value  in  specifying  the  essential  requirements  of  a 
comple;;  C3  situation  and,  thus,  forming  a  speci  f  i  cati  on  for 
TDS,  in  the  course  of  its  construction  and,  later,  its 
experimental  trials  in  the  laboratory  which  yielded  several 
interesting  results. 


From  the  outset  it  was  recognised  that  TDS  should  be  and 
was  an  elaborate  modelling  environment  for  CASTE  used  in  a 
tutorial  or  training  mode.  Using  the  objects  and  actions 
that  are  the  elements  of  TDS  it  is  possible  to  demonstrate 
and  to  explain  a  large  number  of  models  of  many  different 
kinds;  for  example,  of  manouevers  by  one  or  many  craft,  of 
tactics  and  the  like.  Similarly,  the  expertise  of  competent 
users  can  be  represented  in  Lp,  as  one  or  several  entailment 
meshes  which,  if  unfolded,  give  rise  to  plans  or  the 
performance  of  tactics.  This  act  may  be  the  reaction  to  an 
emergency  (for  instance,  that  a.  craft  is  i  ncapac  l  tated  due  to 
lack  of  fuel  or  that  a  trading  route  is  disrupted);  An 
automatic  reaction  if  the  users  are  unable  to  deal  with  the 
emergency  in  question. 


There 
system,  if 


are  several  unique  aspects  of  the  team  decision 
contrasted  with  other  C3  simulations. 


(a)  Each  commander  (or  user)  has  the  role 
of  a  mercenary  able  to  compete  and/or  to 
cooperate  in  the  pursuit  of  a  common 
mission  which  is  to  minimize  the  density 
of  "intruders"  or  "marauder^"  that  enter 
an  environment  and  gradually  converge 
upon  trading  routes  wb?re  they  are 
destined  to  demolish  tradir  vessels  upon 
which  the  viability  of  this  environment 
depends. 
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(b)  Each  mercenary  commander,  or  user,  is 
required  to  divide  attention  since  each 
one  controls  at  least  2  era- ft  by 
assigning  them  tactics  that  are 
subsequently  executed  (-failing 
i nterrupti on )  so  that  the  craft  are  quasi 
autonomous  (in  fact,  2  mi croprocessors 
running  quasi  independently  in  a 
simulated  environment  of  trade,  economy, 
geographi cal  1 y  located  bases  and  so  on). 

(c)  Each  craft,  instructed  by  a 
commander,  acts  upon  partial  information 
about  the  environment.  This  partial 
information  is  displayed  to  the 
commander/user  to  whom  it  belongs, 
through  a  local  display,  craft  at  centre, 
periodically  updated. 


(d)  All  actions,  such  as  moving  a  craft 
or  using  it  to  demolish  an 
intruder/marauder  has  a  cost,  in  terms  of 
fuel  or  some  other  conserved  quantity. 


(e)  The  possible 

obtaining  up  to 
information  about 

environment.  The 


actions  include 
date  but  specific 
the  simulated 
cost  required  is 


deducted  from  the  fuel 
craft  in  question.  At  a 
level  the  craft  is  lost. 


available  to  the 
critical  minimum 


(f)  The  overexpendi ture  of  energy/fuel  by 
a  commander /user  in  demotfl  i  shi  ng  an 
intruder  leads  to  a  "hole"  in  the  quasi 
spatial  "environment".  Such  holes  impede 
navigation  of  craft  or  trading  vessels 
and  are  particularly  harmful  to  the 
envi r onmental  economy  upon  which  the 
supply  of  fuel  depends.  In  this  case  a 
frequently  competitive  liason  must  become 
cooperative  since  at  least  a  pair  of 
craft  must  be  placed  in  the  vicinity  of 
the  hole  in  order  to  repair  it. 


Many  of 
i ncorporated 
del i verabl e 
outlined  in 
capable  of 


these  features  have  been 
into  the  del  vered 
TDS  1.0.  All  of  those 
(a)  (£)  of 'Section  are 

i ncorporat i on  (recording  and 
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rerun  o-f  session  included)  under  a 
roposal  -for  an  augmentation  o-f  the 
listing  TDS  1.0  using  the  existing 
configuration  o-f  standard  hardware. 
These  augmentations  are  desirable  in  view 
o-f  their  low  cost  and  their  upward 
compatibility  with  nowadays  standard  but 
more  sophi sti cated  equipment?  hence, 
their  tr ans-f erabi  1  i t y . 
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5.  The  Course  Assembly  System  and  Tutorial  Environment 
C/CASTE/ 1.0 


It  proved  difficult  to  implement  a  satisfactorily 
transferable  and  transportab 1 e  version  of  the  original  CASTE. 
In  part,  this  is  due  to  the  relative  inflexibility,  for  this 
purpose,  of  standard  computing  equipment.  To  a  lar'ge  extent 
that  barrier  has  been  levelled  out  by  currently  available 
computers  and  more  versatile  peripheral  equipment.  In  part, 
however,  the  difficulty  was  exacerbated  by  persistent 
misperceptions;  perhaps  the  most  irritating  is  crystallised 
in  the  phrase  "text  explanation";  namely,  a  string,  with  luck 
"a  description"  but  seldom  explaining  anything,  let  alone  in 
a  manner  that  can  be  evaluated. 

The  history  of  these  misperceptions  is  recorded  in  my 
files.  They  are  neither  a  sign  of  wilful  glossing  nor  of 
ignorance  but  interesting  exemp 1 i f i cat i ons  of  how  hard  it  is 
to  communicate  some  very  simple  irieas#once  that  they  are 
mistakenly  conceived  to  be  complex  and  ineluctable  ideas. 

A  version  of  CASTE,  stripped  of  the  just  -  noted  - 
mi spercept i ons  was  demonstrable  at  quite  an  early  stage.  It 
was,  in  fact,  demonstrated  during  a  site  visit.  But  it  had 
several  unacceptable  (although  not  -  too  -  obvious) 
i mperf ecti ons  upon  deeper  scrutiny  (for  the  record,  it 
consisted  in  modelling  facilities  in  quasi  independent 
computers  attached  to  the  at  -  that  -  stage  existing 
implementation  of  THOUGHTST I CKER  1.0).  Bringing  the 
imperfections  into  focus,  the  system  with  its  microprocessors 
interacting  through  CCS  communication  cards  only,  was  "hard 
programmed"  (by  analogy  with  "hard  wired")  to  operate  with 
models  in  one  and  only  one  subject  matter  domain.  It  was, 
furthermore,  inherently  unreliable  due,  in  part,  to  hardware 
deficiencies  and,  in  part,  to  lack  of  a  hard  disc 
communication  system  such  as  the  CORVUS  and  OMNINET  acquired 
1 ater . 

The  delivered  CASTE  is,  at  least  in  principle,  devoid  of 
these  flaws.  It  has  a  very  comprehensive  recording  facility 
(provided  that  printout  of  history  #  is  obtained  at  each 
session,  quite  rationally  only  the  current  status  of  a  user 
is  retained,  otherwise).  It  is  chiefly  due,  as  are  several 
improvements  in  the  original  1979  design,  to  Charles  Small 
and  his  colleagues  who,  in  fact,  based  their  design 
specification  upon  an  intelligible  Lp  mesh  representation. 
The  system  runs  with  perfectly  acceptable  rapidity  on  the 
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CORVUS,  Omninet  and  a  minimum  of  3  Apple  2e  microprocessors, 
together  with  random  access  slide  (or  random  access  tape  or 
video  disc)  and  dot  matrix  printer  peripherals.  It  is  very 
reliable  and  offers  compr  ehensi  ve  model  configuring 
capabilities  for  arbitarily  selected  subject  matters.  File 
space  is  readily  assigned  to  different  users.  The  software 
architecture  uniquely  captures  the  architecture  of  the  most 
advaanced  original.  It  is  reliable  and  well  documented  in 
Jaworski *s  ABL/W/4  as  are  THOUGHTST I CKER  1.0  and  TDS  1.0. 

My  colleagues  and  I  have  proposed,  in  concert  with  this 
software  engineering  designer,  upgrading  of  C/CASTE/1.0  to 
CASTE  1.1  and  CASTE  1.2  which  have  been  developed 
al gori thmi cal  1 y  but  not,  as  yet,  implemented.  These  are 
upward  compatible  with  more  advanced  technology.  These 
augmentations  go  as  far  as  practicably  possible  to  upgrade  a 
very  complex  system,  such  as  CASTE,  within  the  limits  of  8 
bit  mi croprocessors  and  include  a  proper  handling  of 
analogies  and  of  general i sati ons;  also,  the  more  important 
types  of  "negation"  or,  more  accurately,  of  "exclusion" 
principles  which  have  been  developed  in  our  theoretical  work. 
Finally,  automatic  model  conf iguration  by  an  author  mode  user 
or  team  of  users  has  been  extended  beyond  the  compass  of  the 
present  options. 
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6.  Stylistic  Tests 


Due,  in  large  measure,  to  my  unavoidable  absence  -for 
several  bits  of  major  surgery  in  the  U.K.  (where  I  have  a 
private  insurance  policy  which  allows  for  rapid  surgical 
processing)  certain  curiosities  have  beset  the  development  of 
the  promised  tests  for  learning,  conceptual  and  innovative 
style.  In  fact,  an  earlier  version  of  a  stylistic  test  (an 
experimental  version  retransl ated ,  in  fact,  from  its  Dutch 
translation  and  from  its  Italian  translation)  has  been 
programmed  for  automatic  admi ni str at i on .  In  fact,  also,  the 
most  recent  version  of  this  stylistic  test  had  already  been 
delivered,  readjusted  for  use  with  a  North  American 
population;  the  evidence  strongly  indicates  that  even  the 
best  European  version  gives  inadequate  emphasis  to  the 
creative  or  inventive,  (in  contrast  to  the  rote  recall), 
intention  of  the  test.  The  cleaned  up  and  "tuned"  to  induce 
a  properly  creative  "mental  set  or  attitude"  version  has  been 
delivered  already  in  a  manually  administered  form.  A 
computer  program  which  performs  the  onerous  chores  of  scoring 
and  of  retaining  full  information  in  raw  score  vectors  (so 
that  various,  equally  valid,  scoring  schemes  can  be  used 
later)  is  a  delayed  but  forthcoming  delivery.  The  manual 
test  is  "SPC2",  the  programmed  form  is  "SPCF'2". 

The  purpose  of  the  tests  is  to  determine  an  anticipated 
style  (to  parameterise  tutorial  strategies  in  CASTE  or  to 
predict  types  of  creative  and  leadership  potential,  notably 
in  TDS) .  There  are  other  possible  applications  and  a  recent 
proposal  to  the  API  concerns  hygenisation  and  delivery  of  the 
manual  form  of  a  companion  test  which  has  known  validity  and 
reliability  in  order  to  employ  "stylistic  tests"  as  part  of 
"before  and  after"  research  designs  or  in  training 
eval uat i on . 

Because  of  the  confusion  it  is  essential  to  distinguish 
between  SPC2  and  SPCF'2  (which  are  compatible  with  a  very 
large  body  of  work,  chiefly  in  Europe  by  several  independent 
researchers  often  using  different  but  equally  defensible  and 
i ntercomparabl e  scoring  schemes,  extending  over  some  12 
years)  and  other  forms  which,  doubtlqss  have  merit,  but  are 
apparently  incompatible  with  a  North  American  population. 
These  other  forms  also  suffer  from  an  inflexible  scoring 
scheme  and,  unless  special  precautions  are  taken,  would  be 
difficult  to  rescore  as  desired. 
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7.  Some  Rules  Not  Mentioned  in  the  Documentation  o-f  Systems 


In  the  context  o-f  "CASTE  and  THOUGHTSTICKER"  it  is  noted 
in  the  documentation  and  earlier  in  this  report  that  any 
topic  or  "shared  concept"  must  be  attached,  directly  or 
indirectly,  to  a  working  model.  However,  it  is  well  worth 
underlining  this  essential  point.  ' 

In  addition,  a  -few  more  authoring  rules  are  needed  to 
render  a  costly  and  powerful  system  free  of  its  hardware 
limitations.  If  these  rules  are  satisfied  then  CASTE  and 
THOUGHTSTICKER  become  valuable  tools  of  a  novel  technology. 
If  not,  the  best  authoring  possible  will  fail  to  provide  the 
user  with  an  image  of  the  internal  organi sati on  and  these 
systems  will  not  live  up  to  the  claims  made  for  them. 

These  rules  are  very  commonsensi cal  if  the  underlying 
theory  is  kept  in  mind  but  did  not  become  obvious  to  me  (a 
primary  theorist)  until  it  was  possible  to  use  the  most 
recent  version  (that  is,  C/CASTE/ 1.0  as  delivered)  in  order 
to  author  and  scrutinise  actual  material,  for  example, 
statistics) . 

Hence,  apart  from  the  already  noted  condition,  that  any 
shared  concept  (or  topic)  must  be  associated  with  one  or 
usually  many  executable  model  for  demonstration,  simulation 
and  evaluation,  the  followinc  restrictions,  chief lv  if  not 
entirely  due  to  hardware  limitations,  are  mandatory  (possible 
avoidence  mechanism  are,  sc  -fa>  as  possible),  noted  below. 

C'CAETE/1.0  operates  on  a  master  CASTE  machine,  an  aim 
and  a  topic  machine.  Tine  aim.  and  topic  machines  act 
independently,  except  for  interrupt  and  the  passage  of 
necessary  si  anal s. 

(1)  It  must  bo  possible  for  a  user  to  compare  and  to 
contrast  and  independently  manipulate  the  aim  and  topic 
machines.  Due  to  screen  capacity  and  sensible  visibility  the 
comparison  and  contrast  must,  to  the  user,  be  simultaneous. 

Go,  for  example,  if  a  written  description  is  displaced 
via  these  machines  (an  alternative  wav  is  a  hicher  definition 
slide  display),  then  this  description  cannot  occupy  more  than 
one  screen  of  text  or  low  definition  graphics. 

(2)  The  same  comment  applies,  but  without  the  slide  option, 
to  demonstr at l on ,  explanation,  simulation  or  evaluation  use 
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of  models. 

'•a. 

(31  One  ercepti  on  is  possible;  topic  models  may  reasonably 
segmented,  i  f  necessary,  into  mutually  comparable  parts. 
This  is  a  difficult  and  not  initially  recommended  process;  it 
places  a  considerable  burden  upon  the  author  and  unless  the 
segments  are  very  clearly  indexed,  is  likely  to  fail. 

<<->  To  each  shared  concept  or  model  there  must,  whatever 
else,  be  a  slide  showing  the  immediate  neighborhood  in  a 
imesn ,  of  the  topic  in  question. 

(5)  There  must  be,  available  to  the  user,  an  image  and 
preferably  a  graphic  image,  of  the  entire  entai lment  mesh. 


(6>)  Anticipating  the  proposed  systems  CASTE  1.1  and  CASTE 
1.2.  the?  assignments  of  the  aim  and  topic  machines  may  be  to 
accept.  models  for  an  aim  topic  or  shared  concept  and  some 
other  topic  model  in  the  cluster  that  any  user  can  choose  or 
in  tie  case  of  CASTE  1.1  and  CASTE  1.2  the  topic  models 
related  tv  an  analogy  or'  a.  general  i  sat  i  on .  In  fact,  (in  this 
case)  th«  differences  and  similarities  are  displayed  on  the 
Master  CASTE  machine.  As  a  matter  of  fact,  there  is  not  much 
differ  ir.ee  (at  any  rate  from  a  users  ooint  of  view).  But, 
obvious) v,  the  business  of  comparison  and  contrast 
simultaneously  is  b roue hi  into  focus. 


(7)  Topic  names  are  best  restricted  to  at  most  <-C 

characters ,  otherwise  overwrit: no  is  likely  to  occur. 

(E)  Usually,  restriction  (7)  is  of  small  consequence. 

However ,  I  have  found  it  nonsensical  in  team,  authoring  where 
one  author.  sav  A,  "thinks  this...."  and  another  .  say  B, 
wishes  to  say? "thinks  that...",  where  "this"  ana  "that"  are 
1  e  n  q  t  h '  , 


Such  statements  can  and  must  be  truncated, 


These  rules  became  evident  to  me  only  after  using  the 
demonstrative  materials.  Many  of  the  topics  in  these 
(including  those  authored  by  myself)  fail  to  satisfy  the 
basic  conditions  set  out  (althouah,  by  slight  rearrangement, 
they  could  readily  do  so). 
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P.  Immedi  ate  Proposals  for  development  of  the  existing 

systems 


Since  the  proposals  concerned  have  been  submitted  in 
draft  form  and  their  receipt  has  been  acknowledged,  it  would 
be  verbose  to  recapitulate  their  content.  It  is,  however, 
worth  noting  that  they  are  3  in  number,  (a),  (fc>>  and  <c> 
bel ow. 

(a)  A  proposal  to  augment  C/CASTE/1.0  into  CASTE  1.1 
which  embodies  practicable  handling  of  analogy  and 
general i sat i on  as  well  as  most  forms  of  negation  or 
exclusion;  later,  into  CASTE  1,2  which  enhances  construction 
of  mere  cornier:  models  and  their  conf  iquraticn  in  an 
appropr iate  CASTE  compatible  form.  The  last  augmentation  is 
accompl  i  shed  by  using  Jc.worsf.i  's  ABL/W/4  as  a  result  of  which 
any  rational  person,  neither  having  nor  desiring  fluency  in 
Microsoft  BASIC  5.3  1=  able  to  manufacture  models  that  are 
specified  in  rational  1  o.nguage  and  transformed  into  code 
acceptable  by  the  e  istina,  or  other  systems. 

Enhancement  cf  THI USHTSTI CK'EF:  1.0  in  a  similar  fashion, 
f o-  f ac.i  1 5  usrc  fc  experienced  authors  or  teams  of  them. 

<b.  Upgrading  TDS  1.0  into  a  system  with  proper 
reso-rii no  and  rspiav  capabilities  and  a  proper  incorporation 
of  -cacti cs.  I r  addition,  it  is  proposed  to  provide  more 
elaborate  and  user  friend! v  display  facilities. 

(c)  The  construction  of  a  manual ly  administered  and  a 
compute  *  ad  mi.  r,i  stored  version  of  a  "stylistic  test"  or 
"Quantifiable  learninc.  innovation  and  recall  system:  SR‘C2" 
which  is  comps  Li bl  e  with  SPC2  (already  delivered).  Since 
this  test  (often  known  as  the  "Smugglers  Test"  is  based  upon 
an  i  some-phi  c  er.t&i  1  mart  mesh,  the  mutual  validity  with  SPC2 
is  established.  Further,  practical  experience  with  the 
earlier  manual  versicns  has  shown  its  high  mutual  reliability 
over  some  600  respondents.  Hence,  it  can  be  confidently  usee 
in  before  and  after  type  studies  of  conceptual  style, 
especially  if  the  experimental  design  of  SPC2  first  SRC2  next 
arid  vice  versa  is  possible. 


theoretical  work  undertaken  and 


•V.  An  overview  of 

di  ssem  nated 


A  considerable  body  of  theoretical  research  has  been 
performed  under  this  contract.  The  r  ubric  "theoretical  "  is 
meant  to  cover  the  productions  ct  algorithmic  speci f i cati on 
and  ,  -for  example  .  the  algorithms  underlying  the,  proposed 
developments  noted  in  Section  B.  Further,  it  has  been 

possible.  to  provide  at  least  one  embedding  of  Lp  in  the 
topological  domain  and  as  a  result  to  furnish  a  more  readily 
implemented  and  more  powerful  theory  of  knowledge  and 
rational  belief. 

Much  of  the  content  of  the  theoretical  work  is  contained 
in  a  memoradup, ,  circulated  in  September  1985  to  give  an  "in 
house  overview"  and  replete  with  references  to  people. ideas 
or  events.  Theoreti cal  work  prior  to  that  has  been 

published,  with  proper  acknowledgement ,  in  an  invited  review 
pacer  of  Conversation  Theory  and  its  prctologic  Lo  in  the 
journal  ECU  (vul  »  32.  no  i,  pages  3  ~  40).  A  -further 

innovation  "active  context  of  a  learner"  ,  appears:  in  the 
C/C  ACTE/ 1 . 0  ct. -rumen  ie  i  on  . 

The  memorandum  is  in  the  process  of  conversion  bv 
editing,  ref:  nemr-nts  a~d  the  elimination  of  purely  "in  house' 
illusions  into  a  lengthy  paps’-  to  appear  in  tne  International 
J  Mar,  Machine  Studies  which  has  cone  through  several  dr  a- ts. 
hf  these  drafts  the  fins.  orie  va  1  1  be  attached  to  this 
ropo-t .  the  Review  Paper  as  Annex  s  1  and  the  Pinal  Irt. 
J.M.M.8.  paper  as  Annexe  2  both, noted  earlier. 

Here I  very  briefly  summarise  the  essential  discoveries 
which  arc.  wi thou i  except i on ,  open  to  i mp 1 enen lotion. 

In  contrast  to  the  previous  i mace  of  Lp  statements  as 
static  ertai 1 ment  mesh  inscriptions;  (aloeit  animated  bv  such 
unfolding  operations  as  prune  or  selective  prune  or  expand  or 
such  inferential  operations  as  saturate/,  there  is  row  an 
obvious  duality  of  mesh.  and  operation.  The  "static"  mesh 
images  the  "dynamic"  action  of  all  of  these  and  of  several 
other  operations  that  do,  in  combi-nation  an  a  perpetual 
operation,  image  the  application,  producion  and  reproduction 
of  all  stable  personal  concepts  in  a  nurd  and  brain.  In  a 
computer  implementation  employing  present  dav  machines 
Application  Ap  ,  in  anv  distinct  mental  organisation  is 
replaced  by  an  application  of  a.  more  limited  kind:  namely, 
the  execution.  Ex  of,  topics  or  shared  concepts  arid  the 
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cinaloqcjs  prorf/sses  that  ere  response  bl e  for  maintaini  no 
thtri  r  coherence  and  prese-  vino  their  d  x  st  i  n  :t  i  or. .  Tne 

resultinu  picture  thus  resembles  a  flu:  os  activity  with 
attrsetors  and  deflectors.  From.  time  to  time  these 
attractors  and  deflectors  have  maxima.  leading,  on  the  one 
hand,  to  bifurcation  into  parts  or  on  the  other  ha.no  to  a 
directed  behaviour,  emerging  from  a  productive  ano  kinetic 
background . 

It  is  thus  legitimate  to  repard  the  apparently  static 
mesh  as  a  product,  a  somewhat  artificial  product,  of  dynamic 
interplay.  On  the  other  hand,  it  is  possible  to  argue  that 

the  mesh  structure  induces  the  activity.  In  fact,  they  are 
complementary.  If  the-'-e  is  a  mesh  and  if  there  are  attacti  ve 
and  deflective  processes,  then  both  are  needed  for  a  proper 
understanding  cs  this  reality. 

As  refine.  in  Anne-.e  i  and  lr  Annexe  Z'i  th_  s  revised 
perspective  shows  an  "Artificial"  intelligence.  But  as  I 
have  of  ten  noted,  there  is  no  reason  to  use  the  qua! i f i er 
"Artificial  •  in  the  contort  of  this  kind  of  an  intellect. 
Ferhs-a  "non  s.ancu'd"  i  collect,  or  ” human.  or  1  v  if  trainee 
to  ts  hunan  i  ntnl )  e.:t '  .  would  capture-  the  intended  meaning. 
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Other  Del  i  vgrab!  es 


Several  other  i  terns  have  bean  and /or  .-are  in  tne  process 
of  delivery.  These  eve  as  follows: 

\ a '  Short  segments  of  video  tape  mads,  at 
the?  request  of  Dr.  Ed  Johnson.  Tnese  * 
segments  are  intended  for  a  soecific 
audience.  They  consist  in  a  brief 

lecture  bv  Paul  Pangsro  deliberately  (at 
request)  in  a  "talking  head"  and  "poster 
board"  mode;  a  demonstration  of  an.  early 
version  of  Apple  II  +  CASTE,  again  by 
Paul  Pan pare,  also  in  a  requested  style, 
suitable  for  i  ntenpoi  at  i  ori  in  a  verbal 
p  r  e  s  c  n  t  a  i  ion  t.  o  a  s  ma  I  1  group. 

(b':  A  primer,  recuested ,  particularly ,  by 
Dr  -  Fulh  Fhoics  and  an  associ  ated  4  tape 
present  str  or.  cons:  stinc  in  interviews, 

5';;;.-lanat  ions,  d omon strati  or, s  and  the 
i  c ,  directed  bv  Bernard  Carver  with  the 
coDpersti  n  of  ether  st«f  f .  As  in  tne 
case  of  5P22  and  £r'CF2  some  confusion 
a-  use-  £••  the  result.  of  the  contract 
fv ;■  t e  r. a .  o -  a r  d  m y  ow n  rather  I  enathy 
h:  u  i  ali  =  at?  on.  On  the  one  hand,  I 

c:  -  a  i  t  e  d  a  film  / 1  a.  o  s  /  scri  p  t  an  d  s  t  a  r  tec 
w  or  k  c  .  a  r _  rntr  der  i  vet  from  it  ar.d 
ini  ended  tc  auon.sr.i  it  at  a  reminder 
bonk!  el.  ’Jo on  my  return  Bernard  and  I 
work fed  briefly  on  the  script  which  it.  by 
setter  iainnent  or  documentary  standards, 
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also,  David  Mitchell 

had 

been 

i  wor 

k  i  n  a 

on  a.  primer  which 

was 

r  roaring  completion  and  is  now  finished. 
Consequently,  it  was  pointless  to  extract 
an  appropriate  'orm  of  primer  from  my 
script.  For  the  ecot-d,  however ,  I  shall 
attach  a  copy  of  the  "Marked  up "  script 
"as  it  was  left"  and  thus  without 
t i d  v i no. 
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11.  Dissemination  of  f.'n owledge 


From  the  outset  the  importance  of  oi  ssemi  nati  nu  knowledge 
and  -findings  bv  refereed  pacers  book  chapters  and  ccr.ference 
proceedings.  na.s  ai  wa  ys  been  emphasized.  Consequent  1  v  a 
great  deal  o-f  effort  has  been  expended  in  presenting 
careful  1 \  pi acea  publications,  symposia  and  seminars. 


